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June  25,  1993 

MEMORANDUM  FOR  COMMANDER  IN  CHIEF,  U.S.  PACIFIC  COMMAND 

SUBJECT:  Audit  Report  on  the  Worldwide  Military  Command  and  Control  System 
Consolidation  in  the  Pacific  Theater  (Report  No.  93-126) 


We  are  providing  this  final  report  for  your  information  and  use.  It  discusses 
DoD  Hotline  allegations  that  the  consolidation  plan  for  the  Worldwide  Military 
Command  and  Control  System  (WWMCCS)  host  computer  sites  in  the  Pacific  Theater 
is  not  cost-effective. 

A  draft  of  this  report  was  provided  to  USPACOM  for  comment  on 
January  28,  1993.  Comments  on  the  draft  report  were  provided  on  April  7,  1993. 
Based  on  documentation  provided  by  USPACOM  in  response  to  the  draft  report,  a 
recommendation  to  reduce  one-time  consolidation  costs  by  $21,000  was  dropped  from 
this  final  report.  For  the  two  remaining  recommendations,  USPACOM  nonconcurred 
with  Recommendation  1.  and  the  associated  potential  monetary  benefits  and  concurred 
with  Recommendation  2.  but  failed  to  provide  an  implementation  date  for  the 
corrective  action.  Details  on  the  unresolved  issues  are  discussed  in  Part  II  of  the 
report. 


DoD  Directive  7650.3  requires  that  audit  recommendations  and  potential 
monetary  benefits  be  resolved  promptly.  Recommendations  and  potential  monetary 
benefits  are  subject  to  resolution  in  accordance  with  DoD  Directive  7650.3  in  the  event 
of  nonconcurrence  or  failure  to  comment.  Therefore,  the  USPACOM  is  requested  to 
provide  comments  on  the  unresolved  matters  in  this  final  report  by  July  26,  1993. 

We  appreciate  the  courtesies  extended  to  the  audit  staff.  If  you  have  any 
questions  on  this  audit,  please  contact  Ms.  Mary  Lu  Ugone  on  (703)  692-3320 
(DSN  222-3320)  or  Ms.  Cecelia  Miggins  on  (703)  692-2897  (DSN  222-2897). 
Appendix  C  lists  the  distribution  of  this  report. 


Robert  L  Lieberman 
Assistant  Inspector  General 
for  Auditing 
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Project  No.  2RE-8003 

WORLDWIDE  MILITARY  COMMAND  AND  CONTROL  SYSTEM 
CONSOLIDATION  IN  THE  PACIFIC  THEATER 

EXECUTIVE  SUMMARY 


Introduction.  The  Pacific  Command,  through  its  subordinate  commands,  operated 
four  Worldwide  Military  Command  and  Control  System  (WWMCCS)  host  computer 
sites  and  planned  to  establish  a  fifth  site  in  FY  1993.  In  February  1992,  the  U.S. 
Pacific  Command  proposed  consolidating  the  five  host  computer  sites  into  two  sites: 
one  at  the  U.S.  Army  Pacific,  Fort  Shatter;  and  one  at  the  U.S.  Pacific  Air  Forces, 
Hickam  Air  Force  Base. 

Objective.  The  audit  objective  was  to  evaluate  the  validity  of  a  Hotline  allegation  that 
the  U.S.  Pacific  Command's  WWMCCS  consolidation  plan  was  not  cost-effective. 

Audit  Results.  The  allegation  was  not  substantiated  and  we  determined  that  the  plan, 
reflected  in  the  System  Development  Notification  (SDN),  was  cost-effective. 
However,  one-time  consolidation  costs  of  $6,495,310  were  overstated  by  $582,250  and 
annual  operating  costs  of  $2,424,600  for  the  consolidated  sites  were  understated  by 
$209,400.  In  addition,  the  plan  included  about  $1.8  million  to  acquire  computer 
equipment  to  support  WWMCCS  program  requirements  that  were  terminated 
subsequent  to  preparation  and  submission  of  the  plan.  Accordingly,  the  processing 
capacity  of  the  DPS  8000/84  in  the  plan  is  not  necessary  and  should  not  be  acquired. 

Internal  Controls.  Internal  controls  were  not  assessed  in  this  audit. 

Potential  Benefits  of  Audit.  This  report  identifies  $2.4  million  in  funds  that  could  be 
put  to  better  use  in  the  fiscal  year  in  which  the  consolidation  plan  is  implemented 
(Appendix  A). 

Summary  of  Recommendations.  We  recommended  that  the  U.S.  Pacific  Command 
amend  the  SDN  to  delete  the  $1.8  million  for  hardware  for  the  terminated  requirements 
and  to  correct  the  overstatement  of  annual  operation  and  maintenance  savings.  A 
recommendation  in  the  draft  of  this  report  to  reduce  the  one-time  consolidation  costs 
related  to  multiplexers  at  Fort  Shatter  was  dropped  from  the  final  report  based  on 
documentation  provided  in  response  to  the  draft  report. 

Management  Comments.  The  U.S.  Pacific  Command  nonconcurred  with  the 
recommendation  to  delete  the  computer  equipment  for  the  terminated  requirements 
citing  a  recent  reanalysis  of  the  theater-wide  WWMCCS  workload.  USPACOM 
concurred  with  the  recommendation  to  correct  the  operation  and  maintenance  savings. 
Details  on  management's  comments  are  provided  in  Part  II  of  the  report,  and  the  text 
of  the  comments  is  in  Part  IV.  Comments  on  the  final  report  are  requested  from 
USPACOM  by  July  26,  1993. 
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This  report  was  prepared  by  the  Readiness  and  Operational  Support  Directorate,  Office 
of  the  Inspector  General  for  Auditing,  DoD.  Copies  of  the  report  can  be  obtained  from 
the  Secondary  Reports  Distribution  Unit,  Audit  Planning  and  Technical  Support 
Directorate  (703)  614-6303  (DSN  224-6303). 


Part  I  -  Introduction 


Background 


The  DoD  established  the  Worldwide  Military  Command  and  Control  System 
(WWMCCS)  in  1962  as  a  global  command  and  control  system.  The  primary 
mission  of  WWMCCS  is  to  support  the  President  and  the  Secretary  of  Defense 
by  providing  secure  communications  to  transmit  tactical  warning  and 
intelligence  information  for  timely  decision  making.  The  secondary  mission  is 
to  support  the  command  and  control  systems  of  the  unified  and  specified 
commands  and  the  WWMCCS-related  information  systems  of  other  DoD 
Components. 

During  1989  and  1990,  the  Secretary  of  Defense  Management  Review  study 
group  evaluated  DoD’s  automated  data  processing  systems  and  determined  that 
there  was  a  potential  for  greater  efficiencies  and  reduction  of  costs  by 
consolidating  some  automated  data  processing  systems  or  facilities.  Based  on 
the  group's  findings,  the  Joint  Staff  decided  to  apply  the  consolidation  concept 
to  DoD's  command,  control,  and  communications  systems.  In  April  1990,  the 
Joint  Staff  requested  that  the  unified  and  specified  commands  determine 
potential  benefits  of  a  consolidation  of  WWMCCS  facilities.  In  response,  the 
U.S.  Pacific  Command  (USPACOM)  prepared  a  study  of  seven  consolidation 
alternatives  for  its  WWMCCS  host  computer  sites.  In  June  1991,  the 
Commander  in  Chief,  USPACOM,  approved  a  plan  to  consolidate  WWMCCS 
host  computer  sites  to  two  locations  on  Oahu,  Hawaii:  Fort  Shafter,  U.S.  Army 
Pacific;  and  Hickam  Air  Force  Base  (AFB),  U.S.  Pacific  Air  Forces. 

The  consolidation  plan  provided  that  Datanet8  communication  processors 
would  replace  host  computers  at  U.S.  Pacific  Fleet,  Makalapa,  Hawaii,  and  at 
U.S.  Forces  Korea,  Taegu,  Korea,  and  remote  network  processors.  The 
consolidation  plan  also  included  upgrades  to  the  USPACOM  WWMCCS 
computer  and  communications  infrastructure.  The  plan  included  computer  and 
communication  requirements  for  the  Joint  Operation  Planning  and  Execution 
System  (JOPES)  Version  4,  a  major  software  application  that  is  processed  on 
WWMCCS. 

In  a  February  1992  System  Development  Notification  (SDN)  to  the  Joint  Staff, 
USPACOM  submitted  its  proposed  consolidation  plan  for  the  WWMCCS. 
USPACOM' s  proposal  addressed  the  consolidation,  related  data  processing 
architecture,  funding  requirements,  and  associated  savings.  USPACOM 
requested  Corporate  Information  Management  (CIM)  funds  for  the  proposed 
consolidation.  CIM  funds  are  Defense  Agency  Procurement  funds  earmarked 
for  CIM  projects  and  are  provided  by  the  Director  of  Defense  Information, 
Office  of  the  Assistant  Secretary  of  Defense  (Command,  Control, 
Communications  and  Intelligence)  (ASD[C3Ij).  The  Departments  of  the  Army, 
Navy,  and  Air  Force,  and  the  U.S.  Marines  Corps  will  also  contribute  funds  for 
the  consolidation. 
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The  ASD(C3I)  provided  CIM  funds  for  the  USPACOM  WWMCCS 
consolidation  project  in  the  following  amounts. 

Table  1.  CIM  Funds 


Military 

Department 

Army 

Navy 

Air  Force 
Total 


Date  Funds 
Provided 

1/28/93 

3/17/93 

1/28/93 

1/17/93 

1/17/93 


Fiscal  Ye^jr 
First/Last 

1993/1995 

1992/1994 

1993/1995 

1991/1993 

1992/1994 


Fund 

Amount 

$2,482,000 

390,000 

1,203,000 

1,785,000 

96.000 

$5.956.000 


Procurement  funds  are  available  for  3  fiscal  years. 


Objective 


The  overall  objective  of  the  audit  was  to  evaluate  the  validity  of  a  Hotline 
allegation  that  the  USPACOM' s  WWMCCS  consolidation  plan  was  not 
cost-effective. 


Scope 


This  economy  and  efficiency  audit  was  made  from  October  1991  through 
April  1992.  The  audit  focused  on  the  USPACOM' s  consolidation  study  and  the 
February  1992  consolidation  proposal.  We  reviewed  and  evaluated  various 
aspects  of  the  consolidation  plan,  including  communications,  software, 
hardware,  personnel,  budget,  and  facilities.  Documents  dated  from 
1980  to  1992  that  describe  WWMCCS-related  command  and  control  systems 
unique  to  each  Service  were  reviewed.  We  did  not  review  funding  requirements 
other  than  the  CIM  funding  requested  in  USPACOM' s  proposal. 

Except  for  the  standards  pertaining  to  internal  controls  discussed  below,  the 
audit  was  made  in  accordance  with  auditing  standards  issued  by  the  Comptroller 
General  of  the  United  States  as  implemented  by  the  Inspector  General,  DoD. 
The  activities  visited  or  contacted  are  listed  in  Appendix  B. 


3 


Introduction 


Internal  Controls 


Because  the  audit  was  limited  to  the  conditions  contained  in  the  DoD  Hotline 
allegation  and  the  scope  was  limited  to  the  consolidation  proposal  as  reflected  in 
the  SDN  and  supporting  documentation,  the  internal  controls  pertaining  to  the 
WWMCCS  in  USPACOM  were  not  assessed. 

Prior  Audits  and  Other  Reviews 


In  "Review  of  Unified  and  Specified  Command  Headquarters,"  February  1988, 
the  Inspector  General,  DoD,  recommended  that  the  USPACOM  assess  the 
feasibility  of  consolidating  the  WWMCCS  facility  at  the  U.S.  Pacific  Air 
Forces,  Hickam  AFB,  into  the  WWMCCS  facility  at  the  U.S.  Pacific  Fleet, 
Makalapa.  In  response  to  the  recommendation,  the  Commander  in  Chief, 
USPACOM,  stated  that  consolidation  at  Makalapa  was  neither  timely  nor 
reasonable.  However,  the  USPACOM  WWMCCS  consolidation  that  was 
subsequently  proposed  and  presented  in  the  SDN  we  audited  meets  the  intent  of 
the  Inspector  General's  February  1988  recommendation  of  consolidating 
WWMCCS  operations  facilities.  There  have  been  no  other  audits  within  the  last 
5  years  that  specifically  addressed  the  WWMCCS  consolidation. 


Other  Matters  of  Interest 


In  1982,  the  Air  Force  was  designated  the  lead  Service  for  the  WWMCCS 
Information  System  (WIS)  Program  to  replace  WWMCCS  automatic  data 
processing  systems.  In  1989,  the  Defense  Acquisition  Board  terminated  WIS 
and  assigned  the  Defense  Information  Systems  Agency  as  the  executive  agent 
for  the  new  WWMCCS  Automatic  Data  Processing  Modernization  (WAM) 
Program.  The  purpose  of  WAM  is  to  modernize  existing  WWMCCS  standard 
automatic  data  processing  systems  and  related  telecommunications.  The 
primary  focus  of  WAM  is  development  of  the  Joint  Operations  Planning  and 
Execution  System  (JOPES)  software.  JOPES  was  planned  as  an  integrated, 
conventional  command  and  control  system  for  use  by  senior-level  decision 
makers.  Of  the  13  JOPES  versions  developed,  3  (Version  3.3)  are  operational. 

In  March  1991,  the  Defense  Acquisition  Board  and  the  Major  Automated 
Information  System  Review  Council  reviewed  the  WAM  Program.  Based  on 
the  reviews,  the  Under  Secretary  of  Defense  for  Acquisition  decided  to  proceed 
with  the  WAM  Program.  However,  on  July  15,  1992,  the  ASD(C3I) 
recommended  the  termination  of  WAM.  On  September  1,  1992,  the  Under 
Secretary  of  Defense  for  Acquisition  terminated  the  WAM  Program  and 
directed  that  the  ASD(C3I)  develop  new  alternatives  for  future  acquisitions  to 
meet  the  critical  mission  need.  On  October  16,  1992,  the  ASD(C3I)  terminated 
development  and  testing  of  JOPES  with  the  exception  of  Version  3.3. 
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Part  II  -  Finding  and  Recommendations 


Consolidation  Plan  for  Host  Computer 
Sites 

The  USPACOM’s  proposed  consolidation  plan  for  the  WWMCCS  host 
computer  sites  in  the  Pacific  theater  was  cost-effective.  However, 
certain  estimates  of  consolidation  costs  and  savings  from  consolidation 
were  not  accurately  reflected  in  the  System  Development 
Notification  (SDN)  proposed  to  the  Joint  Staff.  Errors  were  made  in 
estimating  costs  for  communications  equipment,  and  leased 
communication  lines.  Additionally,  changes  in  WWMCCS  program 
requirements,  which  occurred  since  the  plan  was  prepared,  had  not  been 
considered.  As  a  result,  the  SDN  overstated  estimated  procurement 
costs  for  the  consolidation  by  about  $582,250;  understated  projected 
annual  operating  costs  of  the  consolidated  sites  by  about  $209,400;  and 
included  $1.8  million  to  acquire  computer  equipment  to  support 
WWMCCS  requirements  that  have  been  terminated. 


Background 


In  June  1991,  the  Commander  in  Chief,  USPACOM,  decided  to  consolidate 
five  WWMCCS  host  computer  sites  at  two  sites:  Fort  Shatter,  the  U.S.  Army 
Pacific;  and  Hickam  AFB,  U.S.  Pacific  Air  Forces.  USPACOM’s 
consolidation  plan  proposed  upgrading  remote  network  processing  sites  and 
improving  WWMCCS  communications  in  the  Pacific  theater.  In 
February  1992,  USPACOM  submitted  an  SDN  that  defined  the  proposed 
WWMCCS  Automatic  Data  Processing  Consolidation  Plan,  operational 
requirements,  CIM  funding  requirements,  and  associated  savings.  USPACOM 
requested  $6,495,310  to  implement  the  proposed  consolidation  plan.  About 
$5,956,000  has  been  provided  to  implement  the  consolidation  plan.  As 
proposed,  the  consolidation  would  reduce  annual  WWMCCS  operating  costs  in 
the  Pacific  theater  by  about  $2.4  million.  Savings  would  primarily  result  from 
a  reduction  in  the  number  of  military  personnel  and  in  equipment  maintenance 
costs.  The  consolidation  plan  included  upgrades  and  modernization  for  the 
WAM  program  at  Headquarters,  U.S.  Army  Pacific,  Fort  Shatter,  and 
Headquarters,  U.S.  Pacific  Air  Forces,  Hickam,  AFB. 


Proposed  Consolidation 


In  June  and  July  1991,  a  DoD  Hotline  complaint  alleged  that  USPACOM's 
proposed  consolidation  to  two  WWMCCS  host  computer  sites  at  Hickam  AFB 
and  Fort  Shatter  was  not  cost-effective.  The  Hotline  allegation  also  questioned 
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the  validity  of  the  estimated  savings.  The  consolidation  proposal  included  cost 
comparisons  of  the  current  WWMCCS  sites  for  facilities,  communications, 
hardware,  and  personnel  to  those  of  the  consolidated  sites  and  estimated  the 
resultant  savings.  We  determined  that  the  SDN  overstated  procurement  costs 
for  communications  equipment  by  $296,950  and  for  Datanet  8  communications 
processors  by  $306,600  for  a  total  of  $603,550.  During  the  audit,  USPACOM 
officials  amended  the  SDN  and  reduced  procurement  costs  by  $582,250.  In 
response  to  the  draft  report,  USPACOM  officials  provided  documentation 
which  showed  that  the  remaining  $21,300  was  not  an  overstatement.  An  error 
in  computing  annual  lease  costs  for  communications  lines  resulted  in  an 
understatement  of  $209,400  in  annual  operating  costs. 

Facilities.  In  August  1990,  when  USPACOM  began  to  study  the  WWMCCS 
consolidation,  three  WWMCCS  host  computer  sites  were  active:  two  on  Oahu, 
Hawaii  (U.S.  Pacific  Air  Forces,  Hickam,  AFB,  and  U.S.  Pacific  Fleet, 
Makalapa)  and  one  at  U.S.  Forces,  Taegu,  Korea.  At  the  time  of  the  study, 
the  U.S.  Army,  Pacific,  was  building  a  WWMCCS  host  computer  facility  at 
Fort  Shafter,  Oahu,  Hawaii,  to  replace  its  remote  network  processing  site.  The 
Fort  Shafter  site  was  in  the  final  stage  of  completion,  and  the  site  at 
Hickam  AFB  had  been  remodeled  in  FY  1990.  Therefore,  the  costs  for  those 
buildings  were  considered  sunk  costs  and  were  not  included  in  the  consolidation 
proposal.  Also,  USPACOM  had  plans  for  establishing  another  host  computer 
site  at  Yokota,  Japan.  USPACOM’ s  WWMCCS  consolidation  study  evaluated 
and  compared  the  facilities  at  the  three  active  host  computer  sites  in  regard  to 
floor  space,  electrical  power,  air  conditioning,  expendability,  administrative 
space,  survivability,  and  reliability.  We  visited  the  facilities  on  Oahu,  Hawaii, 
and  reviewed  the  documentation  on  the  three  active  facilities.  We  concluded 
that  USPACOM' s  comparison  of  the  sites  was  valid. 

Communications.  The  SDN  overestimated  nonrecurring  communication  costs 
by  $296,943.  The  proposed  communications  architecture  was  based  on  circuit 
upgrades  required  for  the  JOPES  Version  4.0  and  other  circuits  needed  to  obtain 
the  desired  communication  connectivity.  The  communication  upgrades  that 
were  required  specifically  for  the  consolidation  were  site-to-site  circuits  between 
Datanet  8  communication  processors  and  between  Datanet  8  communication 
processors  and  the  mainframe.  The  $1,223,250  estimate  for  communication 
costs  included  equipment,  engineering,  installation,  and  Defense  Secure 
Network  2  upgrades.  Equipment  to  connect  communications  lines  included,  for 
example,  multiplexers,  security  devices,  modems,  and  channel  cards.  We 
verified  the  equipment  costs  using  the  "U.S.  DoD  Bulk  Modem  Contract" 
(August  1990  through  August  1991)  and  the  "Communications  Group 
Equipment  and  Maintenance  Orders  Procedures-Statistical  Procedures  of 
Time/Division  Multiplexers"  (Contract  No.  DAA  B07-89-D-M084).  Our  audit 
analysis  showed  that  the  unit  cost  of  some  equipment  was  overstated  in  the 
consolidation  proposal.  Unit  costs  were  overstated  because  "ballpark"  figures 
were  used  for  cost  estimates.  When  we  brought  the  overstatement  to  the 
USPACOM 's  attention,  action  was  taken  to  correct  the  errors. 
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Software.  The  WWMCCS  host  computer  site  at  Hickam  AFB  will  process 
Joint,1  Navy,  and  Air  Force  work  load  requirements,  while  the  Fort  Shatter 
WWMCCS  host  computer  site  will  process  Joint,  Army,  and  U.S.  Forces  Korea 
requirements.  The  JOPES  Version  4.0  was  planned  for  processing  at  both  sites. 
Both  proposed  consolidation  sites  developed  computer  disk  storage  space 
requirements  based  on  the  number  and  size  of  software  applications,  data 
protocol  conversion,  work  space,2  and  future  growth.  We  reviewed  the 
calculations  used  to  determine  disk  space  requirements.  We  determined  that 
total  disk  space  requirements  were  overstated  at  Fort  Shafter  and  understated  at 
Hickam  AFB.  Disk  space  requirements  at  Hickam  AFB  were  understated  by 
two  disk  drives,  at  a  cost  of  $189,900.  When  the  errors  were  brought  to  the 
attention  of  USPACOM  representatives,  they  consulted  with  technical  experts  at 
the  Defense  Information  Systems  Agency  and  decided  that  the  proposed  disk 
storage  space  requirements  for  Hickam  AFB  were  sufficient  and  that  the 
two  additional  disk  drives  would  not  be  added  to  the  SDN.  The  overstatement 
of  disk  storage  space  requirements  at  Fort  Shafter  was  minimal  and  did  not 
affect  the  disk  drive  requirements. 

Hardware.  USPACOM' s  WWMCCS  host  computer  sites  will  operate  the 
Distributed  Processing  System  (DPS)  8000/84,  a  mainframe  processor 
manufactured  by  Honeywell,  Inc.  The  DPS  8000/84  would  increase  computer 
processing  power  by  50  percent  and  would  satisfy  JOPES  Version  4.0 
processing  requirements.  The  computer  hardware  architecture  and 
configuration  at  the  two  proposed  sites  was  based  on  an  analysis  of  processing 
requirements  and  projected  JOPES  requirements.  In  addition  to  Joint  systems, 
the  Service-unique  systems  that  support  the  Service  component  commands 
operate  on  the  USPACOM  WWMCCS  computer  system.  Those  Service-unique 
systems  were  considered  by  USPACOM  in  the  estimate  of  hardware 
requirements  because  the  Service-unique  systems  are  not  planned  for  migration 
off  the  WWMCCS  hardware  until  about  FY  1995. 

Datanet  8  communications  processors  will  provide  WWMCCS  connectivity 
between  sites.  Dual  Datanet  8  processors  will  operate  as  host  communication 
processors  at  Hickam  AFB  and  Fort  Shafter,  Hawaii;  as  remote  communication 
processors  at  Makalapa  and  Camp  H.M.  Smith,  Hawaii;  and  as  remote 
processors  at  Elmendorf  Air  Base,  Alaska,  and  Yokota,  Japan.  Single 
Datanet  8  processors  will  operate  as  remote  communication  processors  at 
Taegu,  Korea,  and  Yongsan,  Korea.  Also,  when  USPACOM  submitted  the 
consolidation  proposal  to  the  Joint  Staff,  the  Space  and  Naval  Warfare  Systems 
Command  (SPAWAR)  had  agreed  to  fund  five  Datanet  8  processors.  Since 
then,  SPAWAR  has  agreed  to  fund  seven  Datanet  8  processors.  After  our 
discussion  with  USPACOM  representatives,  they  adjusted  the 


1  Joint  -  Connotes  participation  by  components  of  more  than  one  Service  of  the 
same  nation. 

2  *  Work  space  -  Connotes  the  portion  of  main  storage  that  is  used  by  a 
computer  program  for  temporary  storage  of  data. 
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consolidation  proposal  to  reflect  the  change  in  funding  source.  USPACOM  also 
adjusted  the  CIM  funding  request  to  reflect  additional  funding  received  from  the 
Navy  for  printers. 

Technical  requirements.  The  Defense  System  Support  Organization  (DSSO), 
a  component  of  the  Defense  Information  Systems  Agency,  is  responsible  for 
providing  technical  support  for  WWMCCS  design,  development,  deployment, 
and  evolution.  In  May  1991,  USPACOM  requested  that  DSSO  evaluate  the 
technical  feasibility  of  consolidating  the  host  computer  sites  at  Fort  Shafter  and 
Hickam  AFB.  The  DSSO’s  analysis  focused  on  hardware  and  communications 
architecture  and  requirements.  USPACOM  submitted  data  from  the  General 
Comprehensive  Operating  Supervisor-8,  Performance  Analysis  Reporting 
System.  The  data  represented  a  normal-to-high  work  load  for  the  host  computer 
sites  at  Hickam  AFB,  Hawaii;  Taegu,  Korea;  and  Makalapa,  Hawaii.  DSSO 
also  evaluated  various  communications  configurations  with  moderate  work 
loads.  Based  on  its  analysis,  DSSO  validated  the  communications  and 
hardware  architecture  and  made  recommendations  on  the  hardware 
requirements.  USPACOM  incorporated  the  technical  recommendations  into  the 
consolidation  proposal. 

Personnel.  Implementation  of  the  proposed  consolidation  plan  will  reduce  the 
number  of  personnel  from  383  to  338.  Each  USPACOM  WWMCCS  site 
developed  its  personnel  requirements,  which  were  then  reviewed  by  a  Personnel 
Committee  working  group.  Subsequently,  USPACOM's  WWMCCS 
consolidation  staff  reviewed  the  personnel  requirements.  The  functional  areas 
considered  were  management,  administrative  support,  finance  and  staffing, 
supply,  plans  and  programs,  communications  support,  hardware  and  software 
configuration  management,  security,  data  base,  applications  software,  systems 
software,  computer  performance  evaluation,  training,  local  area  network, 
computer  operations,  and  user  support.  Our  analysis  showed  that  the  majority 
of  the  planned  personnel  reductions  would  result  from  replacing  the  mainframe 
processors  at  two  existing  host  sites  and  five  remote  network  processors  with 
Datanet  8  processors.  Each  existing  and  planned  host  site  developed  a  staffing 
document  that  showed  the  transition  of  the  number  of  existing  personnel  to  the 
number  of  postconsolidation  personnel.  Savings  in  military  personnel  costs 
were  estimated  at  $1,360,000,  but  costs  for  civilian  personnel  would  increase  by 
$549,400.  Net  annual  personnel  savings  were  estimated  at  about  $810,600. 

Operation  and  maintenance  budget.  USPACOM  underestimated  operation 
and  maintenance  costs  by  about  $209,400.  To  present  the  savings  that  would  be 
achieved  through  consolidation,  each  WWMCCS  site  developed 
preconsolidation  costs,  postconsolidation  costs  for  the  functional  areas  of 
civilian  personnel,  travel,  supplies  and  equipment,  equipment  lease  and 
maintenance,  leased  communications  lines,  purchased  services,  and  military 
personnel.  Our  review  of  those  costs  disclosed  significant  differences  only  for 
leased  communications  lines.  We  found  that  both  preconsolidation  and 
postconsolidation  costs  for  leased  communications  lines  were  underestimated. 
Not  all  of  the  telecommunication  circuits  that  would  be  required  for 
consolidation  were  identified  and  included  in  the  SDN.  We  obtained  a  listing  of 
leased  communications  lines  and  costs  from  the  Defense  Commercial 
Communications  Office  and  compared  those  to  the  leased  communications  lines 
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and  costs  presented  in  the  SDN.  Annual  costs  for  leased  communications  lines 
will  increase  by  $199,200  rather  than  result  in  a  reduction  of  $10,200  as  shown 
in  the  SDN. 

WAM  Program 


The  termination  of  both  the  WAM  and  development  of  JOPES  Version  4.0  will 
significantly  affect  WWMCCS  computer  and  communications  requirements. 
The  communications  lines  architecture  for  the  USPACOM  consolidation  builds 
upon  high-speed  circuit  upgrades  required  by  JOPES  Version  4.0.  Also,  the 
DPS  8000/84  mainframe  processor  and  disk  storage  space  requirements  for  the 
WWMCCS  are  partially  based  on  JOPES  Version  4.0  requirements.  The 
USPACOM  consolidation  study  group  estimated  that  JOPES  Version  4.0  would 
require  67  percent  of  the  processing  capability  of  a  DPS  8000  processor. 
Technical  personnel  at  the  Defense  System  Support  Organization  verified  that 
requirement.  Also,  USPACOM  estimated  that  JOPES  Version  4.0  would 
require  1,200,000  little  links  of  disk  storage  space  for  Hickam  AFB  and 
2,500,000  little  links  of  disk  storage  space  for  Fort  Shafter.  Since  the  WAM 
Program  and  development  for  JOPES  Version  4.0  have  been  terminated,  the 
communications  line  speed  architecture,  mainframe  processor  requirements,  and 
disk  storage  space  requirements  will  change  and  consolidation  costs  could  be 
significantly  lower.  We  estimated  that  at  least  $1.8  million  in  funds  are  not 
needed  and  should  be  deleted  from  the  SDN  for  the  procurement  of  a 
DPS  8000/84  processor  and  associated  peripherals  that  would  have  supported 
JOPES  Version  4.0. 

Conclusion,  Although  the  Hotline  allegations  were  not  substantiated,  the 
USPACOM  consolidation  plan  did  not  accurately  reflect  cost  estimates  and 
savings.  Overall,  the  consolidation  plan,  if  implemented  as  proposed,  will 
lower  personnel  and  operation  and  maintenance  costs.  Overstatements  resulted 
from  estimating  errors,  funding  changes,  and  omission  of  some  costs  for  leased 
communications  lines.  Most  of  the  errors  were  corrected  during  the  audit  and 
USPACOM  submitted  an  amended  SDN.  However,  the  requirements  for  the 
JOPES  Version  4.0  that  have  been  terminated,  and  the  overstated  savings 
remain  to  be  corrected. 

Recommendations,  Management  Comments,  and  Audit 
Response 

We  recommend  that  the  Commander  in  Chief,  U.S.  Pacific  Command 
amend  the  Systems  Development  Notification  to: 

1.  Delete  the  $1.8  million  for  computer  hardware  and  high-speed 
communications  architecture  for  the  JOPES  Version  4.0  requirements  that 
have  been  terminated. 

Management  Comments.  The  Commander  in  Chief,  USPACOM, 
nonconcurred  with  Recommendation  1.,  stating  that  a  Joint  Interoperability  and 
Engineering  Organization  (JIEO)  report,  dated  January  29,  1993,  validated  the 
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WWMCCS  computer  hardware  requirements  identified  in  the  SDN.  The  report 
contained  results  of  a  reanalysis  of  the  theater-wide  WWMCCS  computer  work 
load  that  considered  the  cancellation  of  JOPES  Version  4.0.  To  perform  its 
analysis,  JIEO  officials  reviewed  computer  performance  statistics  on 
3-  to  27-day  periods,  during  1992,  at  the  four  existing  USPACOM  WWMCCS 
host  computer  sites.  JIEO  officials  calculated  the  average  and  maximum  work 
load  at  each  of  the  four  sites  (applying  a  relative  performance  factor  of  1.5  to 
work  loads  at  two  sites)  to  estimate  average  and  maximum  work  loads  for  DPS 
8000  processors  at  the  two  proposed  consolidation  sites,  Fort  Shatter  and 
Hickam  AFB.  Work  loads  included  routine  requirements  that  would  receive  a 
lower  priority  during  periods  of  crisis.  Table  2.  shows  the  workload  estimates 
by  proposed  site  for  a  triple  (three  DPS  8000  processors)  and  a  quadruple  (four 
DPS  8000  processors)  configuration.  (Originally,  a  quadruple  configuration 
was  proposed  to  support  JOPES  Version  4.0  requirements.) 

Table  2.  Work  Load  Estimates 


Site 


Triple 

Average  Maximum 
Percentage 


Configuration 

Quadruple 

urn  Average  Maximum 

_ Percentage 


Fort  Shatter  68  111  53  87 

Hickam  AFB  67  100  52  78 


Audit  Response.  We  reviewed  the  January  29,  1993,  report  cited  in  the 
USPACOM  response.  Our  analysis  did  not  confirm  that  JIEO  validated  the 
need  for  computer  hardware  and  high-speed  communications  architecture  for  the 
terminated  JOPES  Version  4.0  requirements.  Rather,  the  report  confirmed  that 
the  additional  processing  capability  (quadruple  configuration)  for  JOPES 
Version  4.0  was  not  needed. 

The  average  work  load  estimated  for  a  triple  configuration  is  well  below  the 
80-percent  threshold  standard  for  average  work  loads  recognized  in  the  JIEO 
report  and  the  June  20,  1991,  report  by  the  Joint  Data  Systems  Support  Center, 
Defense  Information  Systems  Agency,  on  the  Pacific  Consolidation  Studies. 
We  held  discussions  with  contractor  and  Government  personnel  who  confirmed 
that  the  80-percent  is  a  recognized  standard  threshold  for  an  average  work  load. 

Additionally,  the  JIEO  report  states  that  the  average  work  load  decreases  by 
about  7  percent  if  the  unique  processing  requirements  at  Makalapa,  Hawaii,  and 
Taegu,  Korea,  are  eliminated  after  consolidation.  Table  3.  shows  the  work 
loads  when  the  site-unique  requirements  are  eliminated. 
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Table  3.  Estimated  Work  Load  Without 
Site-Unique  Requirements 

_  Configuration _ 

Triple  Quadruple 

Site  Average  Maximum  Average  Maximum 

Percentage _ Percentage _ 

Fort  Shatter  61  104  46  80 

Hickam  AFB  60  97  45  71 


Furthermore,  work  loads  will  again  decrease  as  the  Service-unique  systems  that 
operate  on  the  WWMCCS  mainframes  begin  to  migrate  off  the  WWMCCS 
systems  beginning  in  FY  1995. 

In  conclusion,  the  JIEO  report  shows  that  a  DPS  8000  triple  processor 
configuration  will  meet  work  load  requirements.  As  stated  in  the  JIEO  report, 
the  estimated  average  work  loads  at  Fort  Shafter  and  Hickam  AFB  are 
substantially  lower  than  the  80-percent  threshold.  Also,  during  periods  of 
crisis,  user  access  to  WWMCCS  will  be  prioritized  to  ensure  that  nonessential 
functions  are  not  processed.  We  maintain  that  our  recommendation  to  delete 
about  $1.8  million  from  the  SDN  is  still  warranted.  We  ask  that  USPACOM 
reconsider  its  position  and  provide  additional  comments  in  response  to  the  final 
report. 


2.  Reduce  the  overstatement  of  annual  Operation  and  Maintenance 
savings  by  $209,400  that  resulted  from  the  understatement  of  costs  for  the 
leased  communications  lines. 

Management  Comments.  USPACOM  concurred  with  Recommendation  2., 
but  did  not  provide  a  completion  date. 

Audit  Response.  We  ask  that  the  USPACOM  provide  either  the  date  or 
estimated  date  of  amendment  of  the  SDN. 
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Appendix  A,  Summary  of  Potential  Benefits 

Resulting  from  Audit 


Recommendation 

Reference 


Amount  and/or 

Description  of  Benefit  Type  of  Benefit 


1. 


2. 


N/A1 


Economy  and  Efficiency.  Funds 
could  be  used  more  efficiently  by 
deleting  requirements  based  on 
JOPES  Version  4.0  from  the  SDN. 


$1.8  million  in  CIM 
project  procurement 
funds  put  to  better  use. 
Citation 

97X/X0300.290X. 


Economy  and  Efficiency.  The 
annual  operation  and  maintenance 
savings  resulting  from  consolidation 
will  be  accurately  reflected. 

Economy  and  Efficiency.  Corrects 
the  estimated  costs  reflected  in  the 
SDN  for  communications  equipment 
and  Datanet  8  processors. 


Nonmonetary. 


$582,250  in  funds  put 
to  better  use,  resulting 
from  an  overstatement 
in  funding  requirements 
in  the  SDN  for  this 
CIM  project.  Citation 
97X/X0300 . 290X . 


1  USPACOM  officials  amended  the  SDN  to  reflect  the  overstated  costs 
identified  during  the  audit.  Because  of  the  USPACOM  initiative  additional 
recommendations  for  corrective  action  are  not  needed  in  this  report.  See  page  7 
for  discussion. 
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Office  of  the  Secretary  of  Defense 

Assistant  Secretary  of  Defense,  (Command,  Control,  Communications  and 
Intelligence),  Washington,  DC 


Joint  Staff 

Director,  Command,  Control,  Communications  and  Computers  (J-6),  Washington,  DC 


Department  of  the  Army 

Director,  Information  Systems  for  Command,  Control,  Communications  and 
Computers,  Washington,  DC 

Director,  Program  Executive  Office,  Army  WWMCCS  Information  System,  Fort 
Belvoir,  VA 


Department  of  the  Navy 

Director,  Naval  Operations  Space  and  Electronic  Warfare,  Command  and  Control 
Systems  Division,  Washington,  DC 

Director,  Space  and  Naval  Warfare  Systems  Command,  Information  Management 
System  Program  Directorate,  Washington,  DC 
Director,  Naval  Electronics  Engineering  Activity,  Pacific,  Space  and  Naval  Warfare 
Systems  Command,  Pearl  Harbor,  HI 


U.S.  Marine  Corps 

Director,  Command,  Control,  Communications  and  Computer  Division, 
Washington,  DC 


Department  of  the  Air  Force 

Deputy  Chief  of  Staff,  Command,  Control,  Communications,  and  Computers,  Strategic 
Systems  Division,  Washington,  DC 
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U.S.  Pacific  Command 

Headquarters,  U.S.  Pacific  Command,  Camp  H.M.  Smith,  HI 
Headquarters,  U.S.  Army  Pacific,  Fort  Shatter,  HI 
Headquarters,  U.S.  Pacific  Fleet,  Makalapa,  HI 
Headquarters,  U.S.  Pacific  Air  Forces,  Hickam  Air  Force  Base,  HI 
Headquarters,  U.S.  Fleet  Marine  Force,  Pacific,  Camp  H.M.  Smith,  HI 


Defense  Agencies 

Director,  Defense  Information  Systems  Agency,  Pacific  Area,  Wheeler  Army 
Airfield,  HI 

Director,  Defense  Systems  Support  Organization,  Defense  Information  Systems 
Agency,  Sterling  Park,  VA 
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/ 


Office  of  the  Secretary  of  Defense 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications  and  Intelligence) 
Assistant  Secretary  of  Defense  (Program  Analysis  and  Evaluation) 

Assistant  Secretary  of  Defense  (Public  Affairs) 

Comptroller  of  the  Department  of  Defense 

Assistant  to  the  Secretary  of  Defense  (Intelligence  Oversight) 


Department  of  the  Army 

Secretary  of  the  Army 

Assistant  Secretary  of  the  Army  (Financial  Management) 
Inspector  General  of  the  Army 
Auditor  General,  Army  Audit  Agency 


Department  of  the  Navy 

Secretary  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management) 

Comptroller  of  the  Navy 

Auditor  General,  Naval  Audit  Service 


Department  of  the  Air  Force 

Secretary  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Air  Force  Audit  Agency 
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DoD  Activities 

Commander  in  Chief,  U.S.  Pacific  Command 

Commander  in  Chief,  U.S.  European  Command 

Director,  Defense  Intelligence  Agency 

Director,  Defense  Information  Systems  Agency 

Director,  Defense  Information  Technology  Services  Organization 

Director,  National  Security  Agency 

Director,  Joint  Staff 

Inspector  General,  Defense  Intelligence  Agency 
Inspector  General,  Defense  Nuclear  Agency 
Inspector  General,  National  Security  Agency 


Non-DoD  Activities 

Office  of  Management  and  Budget  . 

U.S.  General  Accounting  Office,  National  Security  and  International  Affairs  Division, 
Technical  Information  Center 


Chairman  and  Ranking  Minority  Member  of  Each  of  the  Following  Congressional 
Committees  and  Subcommittees: 

Senate  Select  Committee  on  Intelligence 
Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 

Senate  Subcommittee  on  Projection  Forces  and  Regional  Defense,  Committee  on 
Armed  Services 

Senate  Subcommittee  on  Readiness,  Sustainability  and  Support,  Committee  on 
Armed  Services 

Senate  Committee  on  Governmental  Affairs 

Senate  Subcommittee  on  Oversight  of  Government  Management,  Committee  on 
Governmental  Affairs 
Senate  Committee  on  Budget 
House  Permanent  Select  Committee  on  Intelligence 

House  Subcommittee  on  Oversight  and  Evaluation,  Permanent  Select  Committee 
on  Intelligence 

House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Services 

Subcommittee  on  Readiness,  House  Committee  on  Armed  Services 

House  Committee  on  Budget 

House  Committee  on  Government  Operations 

House  Subcommittee  on  Legislation  and  National  Security,  Committee  on 
Government  Operations 
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U.S.  Pacific  Command  Comments 


Filial  Report 
Refereac 


COMMANDER  IN  CHIEF,  U.S.  PACIFIC  COMMAND 

(USCINCPAC) 

CAMP  H  M  SMITH,  HAWAII  96961-5025 

J6611 
2021 
Ser:  36° -73 

7  APB  1993 

MEMORANDUM  FOR  OFFICE  OF  THE  INSPECTOR  GENERAL,  AUDIT/READINESS  AND 
OPERATIONAL  SUPPORT  DIRECTORATE 

Subj:  Draft  Audit  Report  on  the  Worldwide  Military  Command  and  Control  System  Consolidation  in 

the  Pacific  Theater 

Ref:  (a)  DoD  1G  Draft  Audit  Report,  WWMCCS  Consolidation  in  the  Pacific  Theater,  28  Jan  93 

1.  The  following  response  is  provided  to  Ref  (a): 

a.  Do  not  concur  with  fourth  and  fifth  sentences,  page  7.  We  were  completing  the  report  as  we 
were  being  audited  Therefore,  to  imply  we  had  not  considered  changes  in  WWMCCS  program 
requirements  is  not  correct 

b.  Page  8,  change  all  references  of  "U  S  Pacific  Air  Force"  to  "U.  S  Pacific  Air  Forces". 

c  Page  12,  second  paragraph,  second  sentence  For  clarity,  recommend  speciiying  the  Hickam 
AFB  and  Ft  Shatter  Datanet  8s  as  host  Datanets,  and  the  others  as  remote  Datanets 

d.  Page  15,  first  paragraph,  first  sentence  For  accuracy,  recommend  you  change  "will"  to 


e  Page  17,  paragraph  1  The  multiplexer  count/cost  at  Ft  Shatter  has  been  relooked  The 
original  SDN  count  of  four  remains  valid.  However,  their  estimated  cost  of  $17,000  each  has  been 
adjusted  Using  the  Arm/s  Bill  of  Materials  Automated  Processing  System  at  HQ  Information  Systems 
Engineering  Command,  the  actual  figure  we  are  using  to  procure  the  multiplexers  is  $13,007  16  each 

f  Page  17,  paragraph  2  Do  not  concur  On  8  January  1993  we  requested  JIEO/TEW  reanalyze 
our  theater-wide  workload  in  light  of  potential  changes  in  our  processing  requirements  (primarily  due  to 
cancellation  of  JOPES  Version  4)  We  felt  it  necessary  to  revalidate  our  hardware  requirements  based  on 
the  ongoing  changes  in  the  W  AM/ JOPES  programs  As  we  did  with  the  original  SDN,  we  coordinated 
the  necessary  PARS-8  data  collection  from  all  sites  for  JIEO’s  analysis  They  conducted  their  reanalysis 
and  provided  a  report,  WWMCCS  Consolidation  CPU  Analysis,  29  January  1993,  which  validated  the 
hardware  requirements  identified  in  the  SDN. 

g.  Page  17,  paragraph  3  Concur 

2  HQ  USCINCPAC  POC  is  Major  Vaughn,  J66 1 1,  DSN  3 15-477-2945. 

Very  respectfully,  / 

\  /  j  /'/  ,  ^ 

JOHNF  O'SULLIVAN, 

/  Colonel,  USAF  / 

, '  Acting  Director  for  Command  and  Control 
/  and  Communications  Systems 


Audit  Team  Members 


William  F.  Thomas 

Mary  Lu  Ugone 
Cecelia  Miggins 
Judy  Curry 
Mark  Ives 
Don  Pierro 
Pamela  Smith 
Darwin  Webster 
Nancy  C.  Cipolla 
Michelle  Johnston 


Director,  Readiness  and 
Operational  Support  Directorate 
Program  Director 
Project  Manager 
Auditor 
Auditor 
Auditor 
Auditor 

Software  Engineer 
Editor 

Administrative  Support 


INTERNET  DOCUMENT  INFORMATION  FORM 


A .  Report  Title:  Worldwide  Military  Command  and  Control  System 
Consolidation  in  the  Pacific  Theater 


B.  DATE  Report  Downloaded  From  the  Internet:  04/24/99 


C.  Report’s  Point  of  Contact:  (Name,  Organization,  Address,  Office 
Symbol,  &  Ph  #):  OAIG-AUD  (ATTN:  AFTS  Audit  Suggestions) 

Inspector  General,  Department  of  Defense 
400  Army  Navy  Drive  (Room  801) 

Arlington,  VA  22202-2884 


D.  Currently  Applicable  Classification  Level:  Unclassified 

E.  Distribution  Statement  A:  Approved  for  Public  Release 

F.  The  foregoing  information  was  compiled  and  provided  by: 

DTIC-OCA,  Initials: _ VM _ Preparation  Date  04/24/99 


The  foregoing  information  should  exactly  correspond  to  the  Title,  Report  Number,  and  the  Date  on 
the  accompanying  report  document.  If  there  are  mismatches,  or  other  questions,  contact  the 
above  OCA  Representative  for  resolution. 


